Efficient resource provisioning in compute clouds via VM multiplexing

Resource provisioning in compute clouds often requires an estimate of the capacity needs of Virtual Machines (VMs). The estimated VM size is the basis for allocating resources commensurate with demand. In contrast to the traditional practice of estimating the size of VMs individually, we pro-pose a joint-VM provisioning approach in which multiple VMs are consolidated and provisioned together, based on an estimate of their aggregate capacity needs. This new ap-proach exploits statistical multiplexing among the workload patterns of multiple VMs, i.e., the peaks and valleys in one workload pattern do not necessarily coincide with the others. Thus, the unused resources of a low utilized VM can be bor-rowed by the other co-located VMs with high utilization. Compared to individual-VM based provisioning, joint-VM provisioning could lead to much higher resource utilization. This paper presents three design modules to enable such a concept in practice. Specifically, a performance constraint describing the capacity need of a VM for achieving a certain level of application performance; an algorithm for estimat-ing the aggregate size of multiplexed VMs; a VM selection algorithm that seeks to find those VM combinations with complementary workload patterns. We showcase that the proposed three modules can be seamlessly plugged into ap-plications such as resource provisioning, and providing re-source guarantees for VMs. The proposed method and ap-plications are evaluated by performance data collected from about 16 thousand VMs in commercial data centers. The results demonstrate more than 45 % improvements in terms of the overall resource utilization

Towards non-intrusive software introspection and beyond

Continuous verification and security analysis of software systems are of paramount importance to many organizations. The state-of-the-art for such operations implements agent-based approaches to inspect the provisioned software stack for security and compliance issues. However, this approach, which runs agents on the systems being analyzed, is vulnerable to some attacks, can incur substantial performance impact, and can introduce significant complexity. In this paper, we present the design and prototype implementation of a general-purpose approach for Non-intrusive Software Introspection (NSI). By adhering to NSI, organizations hosting in the cloud can as well control the software introspection workflow with reduced trust in the provider. Experimental analysis of real-world applications demonstrates that NSI presents a lightweight and scalable approach, and has a negligible impact on the performance of applications running on the instance being introspected.Accepted manuscrip

Workload adaptive power management . . .

In current computer systems, power dissipation is widely recognized as one of the primary critical constraints. Improving the power efficiency of current and emerging systems has therefore become a pressing challenge and an active research area over recent years. Dynamic, on-the-fly management techniques aim to address this challenge by adaptively responding to the changes in application execution. These application patterns, commonly referred to as “phases”, expose distinct, dynamically-varying and often repetitive charac-teristics of workloads. Dynamic management techniques, guided by workload phase infor-mation, can effectively tune system resources to varying workload demands for improved power-efficiency. This thesis researches new methods to characterize and predict application behavior for a dynamic power management endgoal. Specifically, this work has two major thrusts. First, it explores different approaches to characterize and predict dynamically varying workload power behavior. Second, it discusses runtime management techniques for real systems that can proactively adapt processor execution to varying application execution characteristics

Identifying program power phase behavior using power vectors

Characterizing program behavior carries significant value in various avenues of computer engineering re-search from investigation of future architectures tailored better for emerging applications to OS based dynamic management techniques. Most modern applications exhibit distinctively different behavior throughout their run-times, which constitute several phases of execution that share a greater amount of resemblance within themselves compared to other regions of execution. These execution phases can occur at scales comparable to total program execution, necessitating prohibitively long simulation times for characterization. Due to the implementation of extensive clock gating and additional power and thermal management techniques in modern processors, these program phases are also reflected in program power behavior, which can be used as an alternative means of program behavior characterization for power-oriented research. In this paper we present our methodology for identifying phases in program power behavior and determining execution points that correspond to these phases as well as defining a small set of power signatures representative of overall program power behavior. In our similarity analysis we use power vectors sampled at program runtime with our power estimation setup, which consist of power values for 22 processor sub-components. We define a power similarity metric as an intersection of both magnitude based and ratio-wise similarities and develop a thresholding algorithm in order to partition the power behavior into similarity groups. We illustrate our method-ology with the Gzip benchmark for its whole runtime and characterize Gzip power behavior with both the selected execution points and defined signature vectors.

Detecting recurrent phase behavior under real-system variability

As computer systems become ever more complex and power hungry, research on dynamic on-the-fly system management and adaptations receives increasing attention. Such research relies on recognizing and responding to patterns or phases in application execution, which has therefore become an important and widely-studied research area. While application phase analysis has received significant attention, much of this attention thus far has focused on simulation-based studies. In these cycle-level simulations without indeterministic operating system intervention, applications display behavior that is repeatable from phase to phase and from run to run. A natural question, therefore, concerns how these phases appear in real system runs, where interrupts and time variability can influence the timing and behavior of the program. Our work examines the phase behavior of applications running on real systems. The key goals of our work are to reliably discern and recover phase behavior in the face of application variability stemming from real system effects and time sampling. We propose a set of new, “transitionbased” phase detection techniques. Our techniques can detect repeatable workload phase information from timevarying, real system measurements with less than 5 % false alarm probabilities. In comparison to previous value-based detection methods, our transition-based techniques achieve on average 6X higher recurrent phase detection efficiency under real system variability.

Live, Runtime Phase Monitoring and Prediction on Real Systems with Application to Dynamic Power Management

Computer architecture has experienced a major paradigm shift from focusing only on raw performance to considering power-performance efficiency as the defining factor of the emerging systems. Along with this shift has come increased interest in workload characterization. This interest fuels two closely related areas of research. First, various studies explore the properties of workload variations and develop methods to identify and track different execution behavior, commonly re-ferred to as “phase analysis”. Second, a large complemen-tary set of research studies dynamic, on-the-fly system man-agement techniques that can adaptively respond to these dif-ferences in application behavior. Both of these lines of work have produced very interesting and widely useful results. Thus far, however, there exists only a weak link between these con-ceptually related areas, especially for real-system studies. Our work aims to strengthen this link by demonstrating a real-system implementation of a runtime phase predictor that works cooperatively with on-the-fly dynamic management. We describe a fully-functional deployed system that performs ac-curate phase predictions on running applications. The key in-sight of our approach is to draw from prior branch predictor designs to create a phase history table that guides predictions. To demonstrate the value of our approach, we implement a prototype system that uses it to guide dynamic voltage and fre-quency scaling. Our runtime phase prediction methodology achieves above 90 % prediction accuracies for many of the experimented benchmarks. For highly variable applications, our approach can reduce mispredictions by more than 6X over commonly-used statistical approaches. Dynamic frequency and voltage scaling, when guided by our runtime phase pre-dictor, achieves energy-delay product improvements as high as 34 % for benchmarks with non-negligible variability, on av-erage 7 % better than previous methods and 18 % better than a baseline unmanaged system.

LONG-TERM WORKLOAD PHASES: DURATION PREDICTIONS AND APPLICATIONS TO DVFS

COMPUTER SYSTEMS INCREASINGLY RELY ON ADAPTIVE DYNAMIC\ud MANAGEMENT OF THEIR OPERATIONS TO BALANCE POWER AND\ud PERFORMANCE GOALS. SUCH DYNAMIC ADJUSTMENTS RELY HEAVILY ON THE\ud SYSTEM’S ABILITY TO OBSERVE AND PREDICTWORKLOAD BEHAVIOR AND\ud SYSTEM RESPONSES. THE AUTHORS CHARACTERIZE THEWORKLOAD\ud BEHAVIOR OF FULL BENCHMARKS RUNNING ON SERVER-CLASS SYSTEMS\ud USING HARDWARE PERFORMANCE COUNTERS. BASED ON THESE\ud CHARACTERIZATIONS, THEY DEVELOPED A SET OF LONG-TERM VALUE,\ud GRADIENT, AND DURATION PREDICTION TECHNIQUES THAT CAN HELP\ud SYSTEMS TO PROVISION RESOURCES